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Researchers assert that the metacognitive nature of think-aloud pair problem solving (TAPPS) improves 
students ’ problem solving by focusing their attention on their own thinking. The purpose of this study 
was to identify and describe oral verbalizations indicating cognitive processes of secondary-level career 
and technical education students who used TAPPS while troubleshooting. A task outcome (successful or 
unsuccessful) was recorded for students who used TAPPS while troubleshooting a small gas engine fault 
on the basis of whether they were able to identify the correct solution to repair the fault. A qualitative 
interpretive approach was used to describe and interpret students ’ thoughts while they were engaged in 
TAPPS. A quantitative analysis was used to compare levels of oral verbalization between successful and 
unsuccessful TAPPS students. The total average percentage of oral verbalizations indicating 
metacognitive thought was 54% for successful students and 52% for unsuccessful students. The content 
of students ’ oral verbalizations revealed that the metacognitive nature of the TAPPS strategy> does not 
improve problem-solving success when secondary-level career and technical education students do not 
possess enough domain-specific knowledge. TAPPS can provide school-based agricultural education 
instructors with a formative assessment tool for checking students thinking and understanding of 
technical information for problem solving. 
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Introduction 

The complexity of emerging technologies in 
agriculture has increased the difficulties that 
technicians might face when attempting to repair 
equipment problems. Employers want 
employees who can identify problems and find 
solutions to those problems (Johnson, 1991). In 
order to make informed decisions about repairs, 
individuals must be able to synthesize 
information about the problem, evaluate 
potential solutions, and execute their plans. 
School-based agriculture programs are in a 
prime situation to help students develop this type 
of higher order thinking. One of the research 
priority areas for the National Research Agenda 
of Agricultural Education and Communication 
(Osborne, 2007) has been to determine what 
teaching strategies most effectively and most 
efficiently yield desired student outcomes with 


particular groups of students. Ulmer and Torres 
(2007) argued that school-based agricultural 
education students should be taught not only 
content, but also how to improve their thinking. 
Cano and Newcomb (1990) recommended that 
school-based agricultural education instructors 
increase the levels of cognition of their students. 
Recent instructional efforts have focused on 
developing students’ abilities to solve real- 
world problems (Technology for All Americans 
Project, 1996). Hill (1997) stated, “It is 
imperative that professionals in the field 
incoiporate problem solving concepts and 
strategies as a significant element in curriculum 
design and implementation” (p. 32). Pasher, 
Bain, Bottage, Graesser, Koedinger, McDaniel, 
and Metcalfe (2007) recommended that teachers 
find opportunities to have students ask and 
answer questions to promote explanations that 
are metacognitive in nature to improve students’ 
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problem solving. Research has led to 
development of several techniques that have 
shown promise for improving student problem 
solving (Dunlosky & Metcalfe, 2009). 

One of these techniques is the use of think- 
aloud pair problem solving (TAPPS) (Lochhead, 
2001). Lochhead pointed out that the goal of 
TAPPS is the eventual development of students’ 
ability to observe and control their cognitive 
behavior, but Glaser (1984) argued that transfer 
of thinking habits from using general strategies 
like TAPPS is limited because of a lack of a 
direct connection between thinking and problem 
solving during learning. Perkins, Simmons, and 
Tishman (1990) argued that general cognitive 
strategies have potential to be helpful in teaching 
problem solving, but only with deliberate effort. 
Salomon and Perkins (1989) concluded that the 
lack of transfer in thinking habits taught in 
general cognitive strategies is linked to the 
reliance on automatic triggering through practice 
rather than thoughtfully detaching strategies 
from one context and applying them to another. 
Results from previous studies indicated that it 
might be reasonable to assume that individuals 
could develop metacognitive thinking through 
training and instruction (Borkowski, Chan, & 
Muthukrishna, 2000; Cardelle-Elawar, 1995; 
Pintrich, 2002; Schraw & Dennison, 1994). 

Theoretical Framework 

The theoretical framework for this study 
revolves around problem solving, the role of 


metacognition, the impact of verbalization on 
thinking, and TAPPS. Davidson, Deuser, and 
Sternberg (1994) described problem solving as 
“the active process of trying to transform the 
initial state of a problem into the desired one” 
(p. 207-208). This behavior is characterized by 
identification, evaluation, and utilization of 
potential solution paths that would accomplish 
the desired end result. The ability to monitor 
and control one’s thinking to accomplish a 
desired goal is central to metacognition 
(Dunlosky & Metcalfe, 2009). According to 
Davidson et al., metacognition aids problem 
solving by helping an individual focus on 
identifying the problem, defining the problem 
space, generating a mental representation of the 
problem, planning how to proceed, and 
evaluating what is known about their own 
performance. 

Nelson and Narens (1990) developed the 
theoretical model of metacognitive thought seen 
in Figure 1. The object-level represents an 
ongoing cognitive task such as problem solving. 
The meta-level contains a model of a person’s 
understanding of the task they are performing 
and the knowledge that is needed to complete 
the task. The model is informed by the 
individual’s monitoring of their progress 
towards completing the task. As the model is 
updated, the individual’s problem-solving 
performance can be controlled to seek new 
information or test a solution. 


Control 



Monitoring 


j- Meta-level 

^J- Flow of information 


j* Object-level 


Figure 1. Metacognition framework. Adapted from “Metamemory: A Theoretical Framework and New 
Findings,” by T. O. Nelson and F. Narens, 1990, The Psychology> of Learning and Motivation, 26, p. 126. 
Copyright 1990 by Elsevier. Adapted with permission. 


Hacker and Dunlosky (2003) suggested that problem solving through oral verbal reports 

having students explain their thoughts during helps invoke metacognitive thinking. Research 
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has supported the use of self-explanations as a 
strategy for improving student problem-solving 
performance (Ahlum-Heath & Di Vesta, 1986; 
Berry, 1983; Stanley, Mathews, Buss, & Kotler- 
Cope, 1989; Stinessen, 1985). TAPPS is a 
method for invoking oral verbalization during 
problem solving with the goal of developing the 
problem solver’s ability to monitor their 
thoughts (Gourgey, 1998). The TAPPS 
procedure involves a student solving a problem 
while a listener asks questions to prompt the 
student to verbalize their thoughts and clarify 
their thinking (Lochhead, 2000). Berardi- 
Coletta, Buyer, Dominowski, and Rellinger 
(1995) found that students who gave reasons for 
their actions during problem solving performed 
superior to students who were silent, asked to 
talk aloud, or asked problem-focused questions. 

Using TAPPS during troubleshooting has 
been shown to double postsecondary career and 
technical education students’ success rate at 
solving small engine faults (Pate et al., 2004), 
but the impact of using TAPPS during 
troubleshooting in school-based agricultural 
education is uncertain. Students may have 
difficulty focusing their verbalization on 
problem-oriented features rather than actively 
clarifying their own thinking. Students’ 
motivation may inhibit their success (Kluwe, 
1982). If students believe they are poor problem 
solvers, they may make fewer attempts to 
monitor and regulate their thinking, which, in 
turn, may lower the number of solutions 
examined (Hacker, 1998). Does TAPPS help 
secondary-level CTE students’ invoke thinking 
that is metacognitive in nature? 

Purpose 

The purpose of this study was to identify 
and describe oral verbalizations indicating 
cognitive processes of secondary-level career 
and technical education students who used 
TAPPS while troubleshooting. 

Objectives 

1. Identify an optimum level of metacognitive 
statements conducive to successful 
troubleshooting. 

2. Describe secondary-level career and 
technical education students’ thought 
processes while using TAPPS to 


troubleshoot a small gas engine compression 

fault. 

Methodology 

Limitations 

Although this study did not measure all of 
the students’ thought processes, the researchers 
sought to ensure credibility of the qualitative 
data by transcribing directly from digital audio 
recordings of students’ verbalizations during 
troubleshooting. Confirmability of the 
transcripts was checked by having a research 
assistant review the transcripts and compare 
them with the audio recordings. Transferability 
of the results from this study is limited by the 
number of students that were available to 
participate due to the limited numbers of CTE 
courses teaching a unit on small engine 
technology, student attendance, laboratory 
space, and amount of available equipment. 

Research Design 

This study design incoiporated a mixed- 
methods approach. A qualitative interpretive 
approach was used to describe and interpret 
students’ thoughts through their oral 
verbalizations while they were engaged in 
TAPPS. A task outcome (successful or 
unsuccessful) was recorded for students who 
used TAPPS while troubleshooting a small gas 
engine fault on the basis of whether they were 
able to identify the correct fault, identify the 
correct engine system affected, and correctly 
describe how to repair it in order for the engine 
to operate. A quantitative analysis was used to 
compare levels of oral verbalization between 
successful and unsuccessful TAPPS students. 

Participants 

The data source for this study was digital 
audio recordings of 16 secondary-level career 
and technical education students from four Iowa 
schools who engaged in TAPPS during a small 
engine troubleshooting task. Students enrolled in 
selected CTE courses dealing with small engine 
technology were purposely selected to be 
participants in this study. 

Procedure 

Before the experiment, the lead researcher 
provided each student with identical instruction 
regarding domain-specific knowledge on 
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troubleshooting small gas engines via a protocol 
adapted from Webster (2001). Ericsson and 
Simon’s (1993) techniques guided recording, 
transcription, and analysis of the verbal 
protocols. Students were randomly assigned to 
use the TAPPS approach while troubleshooting. 
TAPPS students were randomly assigned 
another student in class that was not in the 
TAPPS group to be a listening partner. Each 
listener was trained on the technique required for 
TAPPS questioning. Listening partners were 
asked to encourage problem solvers to verbalize 
their thoughts without giving any hints or 
assisting the problem solver in finding a 
solution. Following Ericsson and Simon’s 
(1993) protocol, students assigned to the 
troubleshooting role completed a TAPPS 
practice session with two nonsensical word 
problems involving the ordering of letters in the 
alphabet and the order of the Great Lakes based 
on their depths. The practice task was designed 
to ensure problem solvers could verbalize their 
thoughts at an adequate level but was considered 
by the lead researcher to be sufficiently 
dissimilar from the troubleshooting task so as 
not to introduce bias into students’ reports. 
Students assigned to use TAPPS were asked to 
troubleshoot a compression fault involving a 
missing valve spring retainer in a small engine. 
No hints were provided about the nature of the 
engine fault by the lead researcher to the 
students, except students were told the fault did 
not involve removal of the cylinder head or 
crankcase cover. This information was provided 
to prevent students from completely 
disassembling the engine. Each student 
troubleshooter was provided with a complete set 
of basic engine repair tools, a digital voice 
recorder, an attached lapel microphone, and a 
45-minute period in which to identify the 
correct fault, identify the correct engine system 
affected, and correctly describe how to repair the 
fault. 

Analysis 

For the analysis of the transcriptions, coding 
categories of oral verbalizations were developed 
a priori based on Ericsson and Simon’s (1993) 
protocol. Responses from the problem solver 
were coded as level-one, level-two, or level- 
three verbal statements. Level-one 

verbalizations were statements describing 
contents of working memory. Level-two 


verbalizations were statements describing 
nonverbal sensory information. As indicated by 
Ericsson and Simon’s protocol level-three 
verbalizations were considered metacognitive 
statements involving planning, monitoring, and 
evaluating. Students’ statements directed at 
judging themselves as performing poorly or well 
were coded as negative self-assessment and 
positive self-assessment, respectively. 
Students’ statements directed at judging the 
activity positively or negatively were coded as 
positive problem assessment or negative 
problem assessment, respectively. Students’ 
verbalizations consisting of information 
irrelevant to solving the problem were coded as 
not on task. The number of oral verbalizations 
in each category was tabulated for students who 
were successful and unsuccessful at the 
troubleshooting task and then analyzed with 
descriptive statistics including frequencies and 
percentages. 

In order to accomplish the second research 
objective which sought to describe students’ 
thought processes while using TAPPS, students’ 
recorded verbalizations were transcribed and 
coded. A volunteer was recruited to assist with 
transcript analysis. The lead researcher 
transcribed the recordings of the TAPPS 
students and then listened to the recordings to 
identify any errors in the transcripts. To ensure 
credibility of the transcripts as stated by 
Ericsson and Simon (1993), the research 
assistant also reviewed the transcripts and 
compared them with the audio recordings. The 
lead researcher instructed the research assistant 
on how to code the transcripts. The lead 
researcher and research assistant independently 
coded each transcript. Interrater reliability was 
87% between the lead researcher and the 
research assistant. To ensure dependability of 
the analysis, intrarater reliability was assessed 
after a period of four days from the initial coding 
of the transcripts. Five transcripts were 
randomly selected to be recoded by the lead 
researcher and the research assistant. Intrarater 
reliability for the lead researcher was 92%. 
Intrarater reliability for the research assistant 
was 90%. Transcripts coded by the lead 
researcher were used for analysis. Statements 
from each coding category were used to describe 
students’ cognitive processes. 
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Results/Findings 

Quantitative Data 

Objective 1 sought to identify an optimum 
level of metacognitive statements made by 
students using TAPPS that would be conducive 
for successful troubleshooting. Sixteen 
secondary-level career and technical education 
students used TAPPS while troubleshooting a 
small gas engine compression system fault. 
Four of the 16 students were successful at 
troubleshooting the compression system fault. 
Average time to completion for successful 
students was 15 minutes (SD = 6.7). 

Unsuccessful students spent an entire class 
period attempting to troubleshoot the engine 
fault. Class periods ranged from 30 to 35 
minutes in length with an average of 31.2 
minutes (SD = 2.7). 

Successful students had slightly higher 
percentages of oral verbalizations in the 
categories of level-one working memory, level- 
two nonverbal sensory information, level-three 
planning, and level-three evaluating. Although 
the average percentages of oral verbalizations 
were similar between groups, there were 
differences between unsuccessful and successful 
students in standard deviations for the categories 
of level-three planning, level-three monitoring, 
and level-three evaluating (see Table 1). 
Successful students had an average of 10 level- 
three planning oral verbalizations (SD = 9.1) 
while unsuccessful students had an average of 
9.8 (SD = 4.1). Successful students had an 
average of 15 level-three monitoring oral 
verbalizations (SD = 8.0) while unsuccessful 
students had an average of 18.3 (SD = 5.5). This 
would indicate a relatively high rate of variation 
in the number of metacognitive statements 
between successful students. Successful and 
unsuccessful students had similar standard 


deviations for level-one working memory oral 
verbalizations. On average, out of the total 
verbalizations for unsuccessful students there 
were higher percentages of oral verbalizations in 
the categories of negative self-assessment (M = 
2.0, SD = 1.7), negative problem assessment (M 
= 3.0, SD = 3.0), and not on task (M= 3.0, SD = 
4.7). 

Table 2 shows frequencies and percentages 
of oral verbalizations for unsuccessful 
secondary-level CTE students who used TAPPS 
(n = 12). The average total number of 
verbalizations for unsuccessful students was 120 
(SD = 56.1). The average rate of oral 
verbalizations per minute for unsuccessful 
students was 3.8 (SD = 1.5). Of all unsuccessful 
students, student H had the highest rates of oral 
verbalizations in all categories except level- 
three negative self-assessment, level-three 
positive self-assessment, level-three negative 
problem assessment, and level-three positive 
problem assessment. If student H is removed 
from the data set, the average total number of 
oral verbalizations given by unsuccessful 
students becomes 105 (SD = 27.0) and the 
average rate of oral verbalizations per minute 
given by unsuccessful students becomes 3.4 (SD 
= 0 . 8 ). 

Table 3 shows frequencies and percentages 
of oral verbalizations for successful secondary- 
level CTE students who used TAPPS (n = 4). 
The average total number of verbalizations for 
successful students was 66 (SD = 32.7). The 
average rate of oral verbalizations per minute for 
successful students was 4.4 (SD = 1.0). 

Patterns of verbalizations shown in Tables 2 
and 3 were relatively equal when completion 
time was accounted for. Level-one working 
memory, level-three planning, level-three 
monitoring, and level-three evaluating 
accounted for the majority of oral verbalizations. 
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Table 1. Average percentages of oral verbalizations by group 


Code 

Successful 2 

M SD 

Unsuccessful 13 

M SD 

Level-one working memory 

40.6 

7.9 

38.7 

8.5 

Level-two nonverbal sensory information 

2.5 

4.0 

1.3 

1.7 

Level-three planning 

10.0 

9.1 

9.8 

4.1 

Level-three monitoring 

15.0 

8.0 

18.3 

5.5 

Level-three evaluating 

29.0 

4.4 

23.7 

6.1 

Level-thee negative self-assessment 

1.2 

1.7 

2.0 

1.7 

Level-three positive self-assessment 

1.0 

2.1 

0.5 

1.5 

Level-three negative problem assessment 

0.0 

0.0 

3.0 

3.0 

Level-three positive problem assessment 

0.0 

0.0 

0.1 

0.3 

Not on task 

1.5 

2.1 

3.0 

4.0 


a « = 4. Average time spent on troubleshooting =15 minutes (SD = 6.7). 
b 77 = 12. Average time spent on troubleshooting = 31.2 minutes (SD = 2.7). 
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Table 2. 

Frequencies and Percentages of Oral Verbalizations for Unsuccessful Students Who Used Think-Aloud Pair Problem Solving 
Student U L2 L3P L3M L3E L3NSA L3PSA L3NPA L3PPA NOT Total 1 
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Note. LI = level-one working memory; L2 = level-two nonverbal sensory information; L3P = level-three planning; L3M = level-three 
monitoring; L3E = level-three evaluating; L3NSA = level-three negative self-assessment; L3PSA = level-three positive self- 
assessment; L3NPA = level-three negative problem assessment; L3PPA = level-three positive problem assessment; NOT = Not on 
task. 

a Average time spent on troubleshooting = 31.2 min ( SD = 2.7). 
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Frequencies and Percentages of Oral Verbalizations for Successful Students Who Used Think-Aloud Pair Problem Solving 
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Qualitative Data 

Objective two sought to describe secondary- 
level career and technical education students’ 
thought processes while using TAPPS to 
troubleshoot a small gas engine compression 
fault. Most often students’ level-one working 
memory verbalizations described their actions as 
they removed or returned parts to the engine. 
When describing their actions, successful 
student A and three unsuccessful students (E, K, 
and B) failed to use correct engine terminology 
to describe the engine parts. For example, 
successful student A stated, “Taking off ssss 
hold on I don’t know what it is yet but I’m 
taking it off.” Unsuccessful student E stated, 
“I’m gonna take off the something I don’t know 
what it’s called so yeah taking this thing off.” 

There were no differences in content of 
level-two nonverbal sensory information oral 
verbalizations between successful and 
unsuccessful students. Across groups, level-two 
nonverbal sensory information verbalizations 
revealed sensations in smell that were attended 
to by students during troubleshooting. 
Successful student D stated, “Ugh that smells.” 
Unsuccessful student H stated, “Smells good.” 

Students’ level-three planning 

verbalizations were directed toward the order of 
tests to be made to the engine. When planning 
their next test, students described what they 
would do but offered little explanation for why 
they planned to conduct those tests. Successful 
student J started troubleshooting by checking the 
spark plug gap without first identifying if a fault 
existed in the ignition system. Unsuccessful 
student C commented that it seemed to have 
pretty good compression so the next thing to 
check was spark. 

The content of level-three monitoring oral 
verbalizations given by students revealed 
shallow analysis of possible solutions. Students 
did not analyze the results of their tests before 
checking other engine components at random. 
Unsuccessful student C remembered to check 
valve springs but upon examination determined 
nothing was wrong with them. Unsuccessful 
student C stated, “Err look at the valve springs 
really quick just to see if anything is wrong there 
with the valve springs look good, there’s nothing 
abnormal about them.” Unsuccessful student 
C’s listening partner asked, “How do you know 


that something’s not wrong with those?” In 
response, unsuccessful student C stated, “Ah um 
they look pretty normal they didn’t look 
anything out of the ordinary so just by the eye 
they looked fine to me.” Afterward, 
unsuccessful student C’s listening partner asked, 
“Could they be warped or disfigured if 
something was wrong?” Unsuccessful student C 
replied, “Yeah, they would but if they, they look 
pretty good to me.” Unsuccessful students’ 
level-three evaluating verbalizations indicated a 
lack of knowledge regarding the troubleshooting 
procedure and the functions of engine 
components. Unsuccessful student P stated, 
“Alright, then compression, crap I forgot what 
the other one’s were alright, compression, crap 
something else and then the carburetor.” Four 
unsuccessful students (F, L, N, and M) identified 
compression as the fault area but failed to 
identify a solution. Of these four students, two 
(M and F) verbalized that they could not 
remember what to check for compression. 
Unsuccessful student M stated, “I don’t even 
remember everything we’re supposed to check 
for compression, so, if, I can’t even remember 
what to check there’s no way I can get it fixed.” 
Unsuccessful student F stated, “There’s like no 
compression... I can’t even remember.” 

Successful students’ level-three evaluation 
verbalizations were focused on making 
judgments relevant to the cause of the engine 
fault based on the result of their engine test. 
Successful student I stated, “Ah, the 
compression system is wrong; the intake doesn’t 
look to be moving.” Successful students often 
verbalized what they had learned from working 
with the engine. These students made note of 
problem characteristics and related them to what 
they had learned. Successful student J stated. 

Well, I don’t know how I’m suppose to fix 
it, but I think I figured the problem out, um 
the spring doesn’t seem to be seated right, 
um, I’m not sure what I’d do to fix springs, 
the other one has a gap right there... It 
doesn’t seem to be compressed, um make it 
so it would be compressed. 

Negative self-assessment, negative problem 
assessment, and not on task verbalizations 
generally were given by unsuccessful students. 


Journal of Agricultural Education 


115 


Volume 52, Number 1, 2011 



Pate & Miller 


A Descriptive Interpretive... 


Unsuccessful student K stated, “I feel like a 
retard.” Three unsuccessful students (K, M, and 
O) explained they did not like being recorded 
while they were working. Unsuccessful student 
M stated, “Umm, cause 1 don’t like this talking 
through it, I’m not a talker anyway.” 
Unsuccessful student B stated, “Thinking I’m 
probably didn’t get this and I’m going to be the 
one failure in the class.” Two students (E and 
H) seemed to view the activity as irrelevant to 
them. Unsuccessful student E stated, “...this is 
stupid I really don’t care about these stupid 
engines...” Unsuccessful student H stated, 
“Subway eat fresh, ha ha we’re talking about 
random bull.” 

Conclusions/Recommendations/Implications 

When work time was accounted for, 
verbalization rates between unsuccessful and 
successful students were similar in all oral 
verbalization categories except level-three 
negative-self assessment, level-three negative 
problem assessment, and not on task. Successful 
students had no level-three negative problem 
assessment, whereas unsuccessful students had 
an average of three verbalizations for level-three 
negative problem assessment. Also, 

unsuccessful students gave almost twice as 
many negative self-assessment verbalizations as 
successful students. This could have been 
caused by students’ frustration with not finding a 
solution toward the end of their troubleshooting 
activity. Students may have been uncomfortable 
talking aloud. This may have affected their 
motivation to complete the activity (Hacker, 
1998; Kluwe, 1982). On average, unsuccessful 
secondary-level career and technical students 
gave twice the amount of not on task 
verbalizations as successful students. This can 
be explained by the number of not on task 
verbalizations given by unsuccessful students F, 
H, and O. These students’ not on task oral 
verbalizations averaged 9% of their total oral 
verbalizations. The remaining unsuccessful 
students’ not on task oral verbalizations 
averaged only 0.7% (SD = 1.2) of their total oral 
verbalizations. 


Objective 1: Identify an optimum level of 
metacognitive statements conducive to 
successful troubleshooting. 

Results did not indicate a minimum level of 
metacognitive statements during TAPPS that is 
conducive to successfully troubleshooting a 
small engine compression fault. This was 
indicated by the relatively high rate of variation 
in the number of metacognitive statements given 
by students. The total average percentage of 
oral verbalizations across the level-three 
planning, monitoring, and evaluating categories 
was 54% for successful students and 52% for 
unsuccessful students. These rates and the 
qualitative data indicate that TAPPS seemed to 
focus secondary-level students’ thinking toward 
a process-oriented approach during 
troubleshooting (Berardi-Coletta et al., 1995). It 
is possible that the relatively short durations of 
the TAPPS instruction was not sufficient to 
affect the amount of metacognitive statements 
given by students. Extending the length of the 
practice with TAPPS may yield different results. 
Perkins et al. (1990) suggested contextualizing 
instruction of general cognitive strategies by 
teaching them in the target domain with 
vocabulary adjusted to suit the target domain. 
Future research should be conducted to 
determine the effect of allowing secondary-level 
students to practice using TAPPS with an engine 
problem before being tested. Additional 
investigation over an increased treatment period 
may provide insight into the effects of TAPPS 
on other outcomes such as students’ self- 
efficacy and motivation. 

Objective 2: Describe secondary-level career 
and technical education students ’ thought 
processes while using TAPPS to troubleshoot a 
small gas engine compression fault. 

The content of students’ oral verbalizations 
indicates the metacognitive nature of the TAPPS 
strategy does not improve problem-solving 
success when secondary-level career and 
technical education students do not possess 
enough domain-specific knowledge. 
Unsuccessful students’ verbalizations in the 
level-three monitoring and evaluating categories 
often were concerned with their level of 
knowledge regarding troubleshooting and small 
engines. These students had difficulty 
remembering the troubleshooting process and 
the proper functions of engine components. 
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Unsuccessful secondary-level career and 
technical education students verbalized 
negatively about their ability or performance and 
the troubleshooting activity. A majority of 
level-three evaluating statements from 
unsuccessful students focused on assessing their 
knowledge of engine principles and 
troubleshooting. Unsuccessful students 

described their level of knowledge as low or 
nonexistent. In contrast, the content of 
successful students’ level-three evaluating 
verbalizations focused on making judgments in 
relation to their monitoring of the effects of their 
engine tests and their evaluation of engine fault 
symptoms. TAPPS can provide school-based 
agricultural education instructors with a 
formative assessment for checking students 
thinking and understanding of technical 
information for problem solving. A 
recommendation for teaching troubleshooting 
with secondary-level CTE students would be to 
ensure that students possess the prerequisite 
knowledge for troubleshooting with the use of 
TAPPS prior to a summative assessment of 
students’ troubleshooting skills. 

It was assumed that curriculum and 
instruction provided by secondary classroom 
teachers regarding engine theory and operating 
principles did not vary between data collection 
sites prior to troubleshooting instruction. 
Students’ concerns about their knowledge level 
could be connected to their rate of 


troubleshooting success. Davidson et al. (1994) 
observed that amount and quality of a problem 
solver’s domain-specific knowledge can be a 
limiting factor in their ability to reach a solution. 
An implication is that students’ knowledge level 
could be connected to the amount of instruction 
they receive and the difficulty of the 
troubleshooting activity. All secondary-level 
students in this study received only one class 
period of troubleshooting instruction. Students 
were given notes and a demonstration on how to 
troubleshoot the air/fuel delivery, ignition, and 
compression systems. Students were told the 
engine needed all three systems to function 
correctly in order to run, and possible faults for 
each system were described to the students. To 
complete the troubleshooting activity, students 
had to identify the system at fault, identify the 
specific engine component that was 
malfunctioning, and correctly describe the 
appropriate repair. However, secondary-level 
students’ knowledge of basic engine principles 
and operating theory was not formally assessed 
before this study. Although this exploratory 
study offers no support for using TAPPS at the 
secondary level, the reader is cautioned against 
making generalizations from this relatively small 
sample of 16 students. This study does not rule 
out the possibility that TAPPS could be useful 
with other secondary-level students, and we 
strongly recommended that future research 
incoiporate a larger sample size. 
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